Release 10.1A: OpenEdge Development:
Programming Interfaces

Invoking socket operations for SSL sessions

For working in an SSL session, OpenEdge provides an additional client-side feature and also presents a potential deadlock condition when the server, but not the client, is connected using SSL.

Identifying the SSL server in a client session

After connecting to an SSL server socket from a client socket object using SSL, the client can identify the server by reading the value of the SSL-SERVER-NAME attribute on the socket object handle. This CHARACTER attribute returns the value of the SSL server’s X.500 Subject name field from the server certificate after any SSL session has been established for a given socket object.

Deadlocking when a non-SSL client socket connects to an SSL server socket

When you open a non-SSL connection from a 4GL client to a 4GL SSL server, you can initiate a situation that can result in an deadlock condition that is unbreakable from with in the application. This condition results from the following sequence of events:

The non-SSL 4GL client connects a socket to a server socket on a 4GL server using the CONNECT() method.
The client immediately follows the CONNECT() method by a READ() method on the socket object.
The 4GL server enables SSL connections using the ENABLE-CONNECTIONS() method.
The server immediately follows the ENABLE-CONNECTIONS() method by a WRITE() method on the server socket object.

After Step 2, the non-SSL client begins waiting for the socket server to send it data. After Step 4, the socket server waits in a read mode (despite the WRITE()) for the client to begin an SSL connection operation. This results in both 4GL sessions waiting for data from the other. For both the 4GL client and server, the application hangs because the 4GL does not provide a socket timeout feature.

There is no way to detect this situation once the deadlock has occurred. However, the basic strategy for handling it is to know ahead of time that the server is running an SSL session. You can use the following techniques to determine the state of the server before continuing from the client:

On the 4GL socket server, use the SET-CONNECT-PROCEDURE() method to handle a CONNECT event asynchronously and watch a manual timer to determine if the 4GL socket server is hanging.
On the client side, you can also use a small procedure to test the connection before continuing with running the application.